Node / ReticulumProjects / MeshChatX.git / files / CHANGELOG.md

Displaying Rendered • View raw • Download

All notable changes to this project will be documented in this file.

• Micron partials: Support for partial content in Nomad Network pages; partial handling in NomadNetworkPage with processing/clearing, dynamic page updates, and MicronParser integration. Tests for partial handling, regex matching, content injection, and state management.

• LXMF quoted replies: Full support for T383838reply_quoted_content in message parsing, sending, and rendering; reply flow in ReticulumMeshChat with quoted content in LXMF message construction.

• Reply in messages: Reply functionality for messages in ReticulumMeshChat.

• Discovery filters and quick actions: Added discovery whitelist/blacklist configuration and per-announce quick actions in Recently Heard Announces (three-dots menu) to allowlist or blacklist announces directly from each card.

• Security tooling: Added T383838eslint-plugin-security; T383838pip-audit and T383838pnpm audit steps in CI; ESLint disable comments for regex patterns in MarkdownRenderer, DocsPage, and ConversationViewer where required.

• Vitest UI: Vitest UI support and configuration updates for frontend testing.

• Lint task: Central T383838lint task in Taskfile to run all linters.

• Translations: Added translations for T383838ingest_paper_message in German, English, Italian, and Russian.

• Build and CI: cx_Freeze build dependencies in build-test workflow; Wine environment setup for Windows builds; multi-architecture build support (e.g. arm64 on Linux/Windows).

• MicronParser: DOMPurify integration and general improvements; DOMPurify initialization in frontend test setup.

• NetworkVisualiser: Level of Detail (LOD) management and icon cache optimization.

• AudioWaveformPlayer: Adjusted height, improved waveform rendering for dark mode, MutationObserver for responsive updates.

• Interface discovery UX: Discovery settings now include whitelist and blacklist fields in interface pages, and discovered announce overlays now show Blacklisted when matching blacklist patterns.

• ConversationViewer: T383838unknown state for message delivery checks; animation direction control for syncing indicator.

• Interface and MessagesSidebar: Component improvements.

• Propagation sync and markdown: Improved propagation sync and markdown rendering.

• Integrity management: Advanced checks and metadata support; identity manager metadata loading and legacy migrator column handling.

• Database backup and health: Backup data-loss guards: baseline file (message count and total bytes) after each successful backup; detection of suspicious state (e.g. DB was non-empty and now empty, or size collapsed); when suspicious, backups written as T383838backup-SUSPICIOUS-*.zip without overwriting good backups, rotation and baseline update skipped; rotation only applies to normal T383838backup-*.zip. Database health checks at app start and close (integrity plus baseline comparison) with logging; issues exposed as T383838database_health_issues on app and in T383838/api/v1/app/info; toast notification when issues are present; websocket message type T383838database_health_warning for real-time alert. Integrity result handling fixed for provider returning dict rows.

• Vite: API and WebSocket proxy configuration; sourcemaps disabled in build.

• Node and tooling: Node.js engine requirement set to >=24; Node 24 in Raspberry Pi install guide; pnpm 10.30.0; Node and pnpm version updates in Dockerfile and workflows; pip 26.0 in Dockerfile; pip install with T383838--no-cache-dir.

• Dependencies: rns 1.1.3, Vuetify 3.12.1, Electron 39.7.0, autoprefixer 10.4.27, axios 1.13.6, Vue 3.5.29, serialize-javascript; ajv removed in favour of fast-json-stable-stringify and json-schema-traverse; various package and lockfile updates.

• CI: pnpm installation step in CI and test workflows; pnpm cache removed from CI/test workflows; ESLint security rule tuning.

• Docs and legal: README and TODO updates; LICENSE copyright holder updated from Sudo-Ivan to Quad4; Docker arch builder removed.

• Frontend: Unit tests for multiple frontend components; AppPropagationSync and ConfirmDialog tests; accessibility tests for keyboard navigation and ARIA labels; MicronParser and NomadNetworkPage tests for partial handling.

• Backend: Backend test refactor. Database backup and health: unit tests for backup baseline, suspicious detection, rotation skip, and health checks (open/close, no baseline, baseline suspicious, integrity fail); property-based test for T383838_is_backup_suspicious; mock test that T383838database_health_issues is set on identity context setup when health check returns issues; tests to ensure health checks do not mistrigger (empty baseline, legitimate empty DB, small DB). Added discovery tests for whitelist/blacklist persistence, filtering behavior, and sanitization.

• Security and fuzzing: Extended fuzzing and security tests for messages and NomadNet browser: WebSocket handlers (nomadnet.download.cancel, nomadnet.page.archives.get, nomadnet.page.archive.load/add, nomadnet.file.download, nomadnet.page.download, lxmf.forwarding.rule add/delete/toggle, keyboardshortcuts.set), NomadNet path variable parsing (convertnomadnetstringdatatomap), archived page lookup, and message get/delete by hash (single and bulk). Ensures robustness against malformed or adversarial input from the mesh.

• Discovery fuzzing/security: Added property-based and security fuzzing tests to validate discovery pattern sanitization and robustness of interface filtering under malformed or adversarial inputs.

• HTTPS/WSS side-sniffing: New tests in T383838test_https_wss_side_sniffing.py to verify that when HTTPS is enabled, the server speaks TLS only on the API/WS port; plain HTTP connections receive no plaintext HTTP response so other local apps cannot sniff MeshChatX traffic; WSS over the same port is verified.

• Scan workflow: Trivy integration moved to unified scan.yml; Docker workflow Trivy exit code adjusted for successful builds; ZIP artifact build step removed from Gitea release flow.

• Advanced Diagnostic Engine:

• Mathematically grounded crash recovery system using Probabilistic Active Inference, Shannon Entropy, and KL-Divergence.

• Deterministic Manifold Constraints: Actively monitors structural system laws (V1: Version Integrity, V4: Resource Capacity).

• Failure Manifold Mapping: Identifies "Failure Manifolds" across the vertical stack, including RNS identity failures, LXMF storage issues, and interface offline states.

• Intelligent Integrity Monitoring:

• Implemented Shannon Entropy Analysis for critical files and databases to detect non-linear content shifts (e.g., unauthorized encryption or random data injection).

• Integrated SQLite Structural Verification via T383838PRAGMA integrity_check to distinguish between binary hash changes (dirty shutdowns) and actual database corruption.

• Refined ignore logic for volatile LXMF/RNS files to eliminate false positives in tampering detection.

• Added advanced security alerts for content anomalies, signature mismatches, and critical component compromises.

• RNS Auto-Configuration:

• Automatic creation and repair of the Reticulum configuration file (T383838~/.reticulum/config) if it is missing, invalid, or corrupt.

• Expanded Security Pipeline:

• Integrated Trivy for both filesystem (codebase) and container image scanning.

• Consolidated security scans into a unified T383838scan.yml workflow for better visibility.

• Updated container workflows to include fail-fast filesystem checks.

• Network Visualiser Optimization:

• Implemented AbortController support to cancel pending API requests on component unmount.

• Added high-performance batch fetching for path tables and announces (up to 1000 items per request).

• Announce Pagination:

• Added backend and database-level pagination for announces to improve UI responsiveness in large networks.

• Improved Installation:

• Added support and documentation for installing via Pre-built Wheels (.whl) from releases, which bundle the built frontend for a simpler setup experience.

• Reliability & Memory Management:

• Fixed a major concurrency issue where in-memory SQLite databases (T383838:memory:) were not shared across background threads, causing "no such table" errors.

• Resolved T383838asyncio event loop race conditions in T383838WebAudioBridge using a lazy-loading loop property with fallback.

• Refactored T383838IdentityContext teardown to ensure all managers are properly nullified and callbacks cleared, preventing memory leaks and reference cycles.

• Added client list cleanup in T383838WebAudioBridge when calls end.

• UI/UX:

• Enhanced LXMF link handling with better rendering logic for T383838lxmf:// and T383838rns:// URIs.

• Fixed a critical hang in the Startup Wizard where "Finish" or "Skip" buttons could become unresponsive.

• Improved UI navigation safety by automatically closing the tutorial modal when navigating away.

• Refined T383838MarkdownRenderer regex patterns to prevent empty bold/italic tags and improved matching for single delimiters.

• Infrastructure & CI:

• Added dedicated build scripts for Arch Linux packaging to handle permissions and T383838makepkg execution.

• Updated Docker dev-image workflows to trigger on master branch pushes.

• Refactored telemetry data packing for more efficient location transmission.

• Updated dependencies including Electron Forge (7.11.1), Prettier (3.8.0), and ESLint plugins for better stability and formatting.

• Testing:

• Frontend UI Test Suite Expansion: Added comprehensive Vitest suites for all diagnostic and utility tools (Ping, Trace, Probe, RNode Flasher, Micron Editor, etc.).

• Property-Based Testing: Significant expansion with T383838hypothesis to ensure robustness of the diagnostic engine, identity restoration, and markdown renderer.

• Integrity Validation Suite: Added extensive property-based tests for entropy mathematical bounds and simulated corruption scenarios (SQLite b-tree breakage, content type shifts).

• Added automated verification for Python version and legacy kernel compatibility diagnostics.

• Configured temporary log directory management for tests to improve portability.

Season 1 Episode 1 - A MASSIVE REFACTOR

• Banishment System (formerly Blocked):

• Renamed all instances of "Blocked" to "Banished", you can now banish really annoying people to the shadow realm.

• Blackhole Integration: Automatically blackholes identities at the RNS transport layer when they are banished in MeshChatX. This prevents their traffic from being relayed through your node and publishes the update to your interfaces (trusted interfaces will pull and enforce the banishment).

• Integrated RNS 1.1.0 Blackhole to display publishing status, sources, and current blackhole counts in the RNStatus page.

• RNPath Management Tool: New UI tool to manage the Reticulum path table, monitor announce rates (with rate-limit detection), and perform manual path requests or purges directly from the app.

• Maps: You can now draw and doodle directly on the map to mark locations or plan routes.

• Calls & Audio:

• Added support for custom ringtones and a brand-new ringtone editor.

• New Audio Waveform Visualization for voice messages, providing interactive playback with a visual waveform representation.

• Paper Messages: Introduced a tool for generating and scanning paper-based messages with built-in QR code generation for easy sharing.

• LXMF Telemetry & Live Tracking:

• Full implementation of Sideband-compatible (Still need to test Columba) telemetry (FIELDTELEMETRY & FIELDTELEMETRY_STREAM).

• Live tracking with real-time map updates, distinct blue pulsing animations, and historical path tracing (breadcrumb trails).

• Mini-chat integrated into map markers for quick communication with telemetry peers.

• Privacy controls with global telemetry toggle and per-peer "Trust for Telemetry" settings.

• Detailed telemetry history timeline with interactive battery voltage/percentage sparkline charts.

• Documentation: You can now read all the project guides and help docs directly inside the app.

• Reliability:

• If the app ever crashes, it's now much better at picking up right where it left off without losing your data.

• Added Identity Switch Recovery: mechanism to restore previous identities or create emergency failsafes if a switch fails.

• Multi-Identity "Keep-Alive": Identities can now be kept active in the background when switching, ensuring you still receive messages and calls across all your personas.

• Added Database Snapshotting & Auto-Backups: You can now create named snapshots of your database and the app will perform automatic backups every 12 hours.

• Added Emergency Comms Mode: A lightweight mode that bypasses database storage and non-essential managers, useful for recovering from corrupted data or running in restricted environments. Can be engaged via UI, CLI flag (T383838--emergency), or environment variable (T383838MESHCHAT_EMERGENCY=1).

• Added Snapshot Restoration: Ability to restore from a specific snapshot on startup via T383838--restore-from-snapshot or T383838MESHCHAT_RESTORE_SNAPSHOT environment variable.

• Diagnostics:

• New Debug Logs Screen: View and export internal system logs directly from the UI for easier troubleshooting.

• Community: Better support for community-run network interfaces and checking TCP ping status of suggested interfaces.

• UI Tweaks: Added a new confirmation box for important actions and a better sidebar for browsing your archived messages.

• Micron Editor: Added multi-tab support with IndexedDB persistence, tab renaming, and a full editor reset button.

• Desktop Enhancements (Electron):

• Multi-Window Calls: Optional support for popping active calls into a focused 2nd window.

• System Tray Integration: The app now minimizes to the system tray, keeping you connected to the mesh in the background.

• Native Notifications: Switched to system-native notifications with deep-linking (click to focus conversation).

• Protocol Handling: Register as default handler for T383838lxmf:// and T383838rns:// links for seamless cross-app navigation.

• Hardware Acceleration Toggle: Power-user setting to disable GPU acceleration if flickering or glitches occur.

• Power Management: Automatically prevents system sleep during active audio calls to maintain RNS path stability.

• Added Web Audio Bridge which allows web/electron to hook into LXST backend for passing microphone and audio streams to active telephone calls.

• Added LXMFy for running bots.

• Added RNS Discoverable Interfaces https://markqvist.github.io/Reticulum/manual/interfaces.html#discoverable-interfaces and ability to map them (ones with a location).

• Blazingly Fast Performance:

• Network Rendering: The Network Visualizer now uses intelligent batching to handle hundreds of nodes without freezing your screen.

• Memory Optimization: Added a smart icon cache that automatically clears itself to keep the app's memory footprint low.

• Parallel Loading: The app now fetches network data in parallel, cutting down startup and refresh times significantly.

• Lazy Loading: Documentation and other heavy components now load only when you need them, making the initial app launch much faster.

• Smoother Settings: Changing settings now uses "smart saving" (debouncing) to prevent unnecessary disk work and keep the interface responsive.

• Backend Efficiency: A massive core refactor and new database optimizations make message handling and search nearly instantaneous. Added pagination to announce and discovery lists to improve performance in large networks.

• Calling: The call screen and overlays have been completely redesigned to look better and work more smoothly.

• Messaging:

• Polished the message lists and archive views to make them easier to navigate.

• Added "Retry All" functionality for failed or cancelled messages in conversation views.

• Improved handling of T383838lxm.ingest_uri.result with detailed notifications for success/error/warning states.

• Maintenance Tools: Added new maintenance utilities to clear LXMF user icon caches and manage backup configurations.

• Network View: The visualizer that shows your network connections is now much clearer and easier to understand.

• Languages: Updated translations for English, German, and Russian. Added Italian (it-IT) localization. Added a toggle to easily enable or disable translation services.

• Search: The command palette (quick search) and notification bell are now more useful.

• CartoDB Tiles - some more styles if OSM is not enough for you, MBtiles will export tiles from the selected one.

• Basic Markdown in Messages - Support for basic markdown in messages

• Fixed issues where switching between different identities could sometimes cause glitches.

• Fixed several small bugs that could cause messages to get stuck or out of order.

• Lots of small UI fixes to make buttons and menus look right on different screens.

• Fixed glitchy message page

• Backend Architecture:

• Decoupled logic into new specialized managers: T383838community_interfaces.py, T383838docs_manager.py, T383838identity_manager.py, T383838voicemail_manager.py, and T383838nomadnet_utils.py.

• Added specialized utility modules: T383838meshchat_utils.py, T383838lxmf_utils.py, T383838async_utils.py, and T383838identity_context.py.

• Implemented a robust state-based crash recovery system in T383838src/backend/recovery/.

• Self-Healing Database Schema: Enhanced T383838DatabaseSchema with automatic column synchronization to prevent crashes when upgrading from older versions with missing columns.

• Enhanced database layer with T383838map_drawings.py and improved T383838telephone.py schema for call logging.

• Standardized markdown processing with a new T383838markdown_renderer.py.

• Added pagination support for announce queries in T383838AnnounceManager.

• Performance Engineering & Memory Profiling:

• Integrated a comprehensive backend benchmarking suite (T383838tests/backend/run_comprehensive_benchmarks.py) with high-precision timing and memory delta tracking.

• Added an EXTREME Stress Mode to simulate ultra-high load scenarios (100,000+ messages and 50,000+ announces).

• Implemented automated memory leak detection and profiling tests using T383838psutil and custom T383838MemoryTracker utilities.

• Full-Stack Integrity & Anti-Tampering:

• Implemented Backend Binary Verification: The app now generates a SHA-256 manifest of the unpacked Python backend during build and verifies it on every startup in Electron.

• Added Data-at-Rest Integrity Monitoring: The backend now snapshots the state of identities and database files on clean shutdown and warns if they were modified while the app was closed.

• New Security Integrity Modal: Notifies the user via a persistent modal if any tampering is detected, with a version-specific "do not show again" option.

• Frontend Refactor:

• Migrated complex call logic into T383838CallOverlay.vue and T383838CallPage.vue with improved state management.

• Implemented modular UI components: T383838ArchiveSidebar.vue, T383838RingtoneEditor.vue, T383838ConfirmDialog.vue, and T383838AudioWaveformPlayer.vue.

• Integrated a new documentation browsing system in T383838src/frontend/components/docs/.

• Added custom Leaflet integration for map drawing persistence in T383838MapPage.vue.

• Infrastructure:

• Added T383838Dockerfile.build for multi-stage container builds.

• Introduced T383838gen_checksums.sh for release artifact integrity.

• Comprehensive Testing Suite:

• Added 80+ new unit, integration, and fuzz tests across T383838tests/backend/ and T383838tests/frontend/.

• Implemented property-based fuzzing for LXMF message parsing and telemetry packing using T383838hypothesis.

• Updated CI coverage for telemetry and network interface logic.

• Updated core dependencies: T383838rns, T383838lxmf, T383838aiohttp, and T383838websockets.

• Developer Tools & CI:

• New T383838task commands: T383838bench-backend (Standard suite), T383838bench-extreme (Breaking Time and Space), T383838profile-memory (Leak testing), and T383838bench (Full run).

• Added Gitea Actions workflow (T383838bench.yml) for automated performance regression tracking on every push.

• Utilize Electron 39 features:

• Enabled ASAR Integrity Validation (Stable in E39) to protect the application against tampering.

• Hardened security by disabling T383838runAsNode and T383838nodeOptions environment variables via Electron Fuses.

• Implemented 3-Layer CSP Hardening: Multi-layered Content Security Policy protection across the entire application stack:

1. Backend Server CSP (T383838meshchatx/meshchat.py): Applied via T383838security_middleware to all HTTP responses, allowing localhost connections, websockets, and required external resources (OpenStreetMap tiles, etc.).

2. Electron Session CSP (T383838electron/main.js): Shell-level fallback CSP applied via T383838webRequest.onHeadersReceived handler to ensure coverage before the backend starts and for all Electron-rendered content.

3. Loading Screen CSP (T383838electron/loading.html): Bootloader CSP defined in HTML meta tag to protect the initial loading screen while waiting for the backend API to come online.

• Added hardware acceleration monitoring to ensure the Network Visualiser and UI are performing optimally.